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ABSTRACT 

A dilemma in designing computer simulations for 
instruction is how to provide a challenging exploratory environment 
and yet provide sufficient support so that students do not become 
lost. Directive support such as corrective feedback may detract from 
the exploratory quality of a simulation. Two methods of non-directive 
support are: (1) simplifying the initial presentation of the 
simulation by having the student manipulate only some of the 
variables; and (2) presenting the simulation problem to a small group 
rather than to individuals so that students may give each other 
feedback on their hypotheses and procedures. This paper examines the 
effect of both of these methods and their interaction upon 
undergraduate students solving a problem in a four variable computer 
simulation of the spread of an epidemic. Subjects were divided into 
four treatment groups working with varying complexity levels. In 
terms of instructional effectiveness, as measured by the percentage 
of students reaching a correct solution, a significantly greater 
proportion of subjects who worked individually reached the correct 
conclusion when the task was initially simplified. In contrast, a 
greater proportion of subjects who worked in groups reached the 
correct conclusion when presented with full initial complexity. 
Subjects performed significantly more simulation runs when they were 
presented fewer variables first and all of the variables later. This 
study suggests that when a complex, multi-variable simulation is 
presented, it may be preferable to present it to groups for solution 
rather than attempt to break it down into component parts. (Contains 
17 references.) (Author/AEF) 
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Abstract 



Computer simulations allow students to explore learning environments by forming hypotheses and 
making and testing predictions. A dilemma in designing such simulations is how to provide a challenging 
exploratory environment and yet provide sufficient student support so that students do not become lost. 

Directive support such as corrective feedback may detract from the exploratory quality of a simulation. Two 
methods of non-directive support are 1) simplifying the initial presentation of the simulation by having the 
student manipulate only some of the variables and 2) presenting the simulation problem to a small group rather 
than to individuals so that students may give each other feedback on their hypotheses and procedures. This paper 
examines the effect of both of these methods and their interaction upon undergraduate students solving a problem 
in a four variable computer simulation of the spread of an epidemic . 

Subjects were divided into four treatment groups: individuals working with reduced initial complexity , 
individuals working with full initial complexity , groups working with reduced initial complexity , and groups 
working with full initial complexity. In terms of the effectiveness of instruction as measured by the percentage 
of students reaching a correct solution, a significantly greater proportion of subjects who worked individually 
reached the correct conclusion when the task was initially simplified In contrast ; a greater proportion of 
subjects who worked in groups reached the correct conclusion when presented with full initial complexity. 

Subjects performed significantly more simulation runs when they were first presented fewer variables 
and finally all of the variables. When a task is complex individuals may benefit from working with a group. In 
this study group work provided no benefit for the simplified task. The study suggests it may be preferable , in 
presenting a complex, multi -variable simulation , to present it to groups for solution rather than attempt to 
break it down into component parts. 

With increasing emphasis on constructivist approaches to learning and greater availability of appropriate 
software, instructional computer-based simulations arc increasingly being used to allow students to explore complex 
multi-variable phenomena by manipulating variables and observing changes. For example, Smithtown allows students 
to vary economic variables (wages and prices) and observe outcomes, such as shift in the demand function (Shute & 
Glaser, 1990). MIDAS, a simulation in the domain of decision-support theory, requires users to see the effects of 
specifying different weights on user preference scores (de Jong, de Hoog, de Vries, 1993). REFRACT allows students to 
see the effects of manipulating variables such as optical density, angle of refraction, and image distance on objects such 
as surfaces, lenses and rays (Reimann, 1991). Numerous such simulations now exist in both the physical and social 
sciences. Common to all of them is the process of experimentation, which includes forming hypotheses, making and 
testing predictions, and modifying the hypotheses, A dilemma is that this process, basic to scientific inquiry, is both a 
prerequisite for and an outcome of using such simulations. Some simulations have been used in an attempt to teach the 
methods of scientific inquiry, for example Discovery Lab (Minnesota Educational Computing Corporation, 1984). But 
more often, as in the examples above, the main purpose is to teach content, namely, the relationships of variables 
defining a physical or social phenomenon. 

While it is widely believed that simulations can effectively teach about complex systems through this controlled 
method of discovery, studies have demonstrated that not all students meet with success. (e.g. Shute & Glaser, 1990; Njoo 
& De Jong, 1991). Learning in such environments requires students to construct their own knowledge and is generally 
very demanding for students, even with some form of instructional support. However, the provision of such support 
needs to be considered in terms of its effects on the nature and level of learners* exploratory strategies. Ideally, such 
support should facilitate student learning while maintaining the exploratory nature of the learning experience. More 
precisely, what is needed are support strategies which solve the dilemma between a low-support ; low effectiveness , high 
involvement and high-support , high effectiveness , low involvement situation. 

Various authors have investigated the effects of varying levels of instructional support in experiential learning 
environments. De Jong, de Hoog, de Vries (1993) categorize such support as either directive (for example, hints, 

Socratic dialogue, corrective feedback) or non-directive (for example, hypothesis sketch pads, goal decomposition trees, 
overviews of output obtained or input history). Non-directive support is preferred as it maintains the exploratory nature 
of the learning environment. Shute and Glaser (1990) provided non-directive support in terms of tutoring in scientific 
skills. Reimann (1991) provided structure for subjects’ hypothesis generation and testing through the use of window and 
notebook facilities to keep track of experiments and to organize and manipulate information, and through the use of 
graphic and verbal feedback. Njoo and de Jong (1991) studied the effects of providing different levels of support to 
college level students of engineering using a simulation in the domain of control theory. In addition to providing 
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subjects with notetaking facilities, Quinn and Alessi (1994) showed some success for a strategy of breaking the overall 
task into subtasks of increasing complexity when used in conjunction with a strategy of generating and testing multiple 
hypotheses. Quinn & McCune (1995) showed a greater level of exploration on the part of subjects who were successful 
in completing a variable optimization task than those who were unsuccessful. 

Our current focus is on optimizing student learning while promoting and maintaining exploratory behavior in 
the context of a complex multivariable simulation. The literature on scientific inquiry provides many guidelines to foster 
hypothesis generation and testing and subsequent exploratory behavior (Popper, 1978; Holland, Holyoak, Nisbett, and 
Thagard, 1986), with recent research identifying several relevant factors. These include task complexity (Mynatt, 
Doherty, & Tweney, 1978; Gorman, 1989), effects of working in groups (Gorman, 1989), structure of the phenomenon 
(Klayman & Ha, 1987), goals (Klayman & Ha, 1987), timing of tests of confirmation and falsification (Mynatt et al, 
1978; Tweney, Doherty, Womer, Pliske, Mynatt, Gross, & Arkelin, 1980), familiarity of the context (Cheng & 
Holyoak, 1985), subjects* prior knowledge (Klahr & Dunbar, 1988), and the number of hypotheses generated (Mynatt et 
al, 1978; Tweney et al, 1980; Klahr & Dunbar, 1988). 

This present study is concerned with the development of non-directive support strategies which take into account 
two of the factors mentioned above - structure of the phenomenon and the effects of working in groups, and is a follow- 
up to Quinn & Alessi (1994). In that study, the phenomenon being investigated depended on several variables and the 
overall task presented to subjects was to investigate the effects of modifying four of these variables. One of the variables 
behaved in a counterintuitive manner (the optimal value for the goal assigned to subjects was its maximum value, but it 
appeared to many subjects that the optimal value should be the minimum value) and this was the principal reason for 
subjects failing to obtain the optimal combination of the four variables. The purpose of this study was to increase the 
proportion of subjects who obtained the optimal combination of variables by requiring subjects to work in dyads. The 
rationale behind this approach was that requiring subjects to work in pairs would increase the probability of subjects 
perceiving the counterintuitive nature of the optimal combination of values, while maintaining the exploratory nature of 
the task. Thus, requiring subjects to work in pairs was seen as a form of non-directive support. In addition, some 
subjects were presented with the overall task initially and some subjects were presented with the task in stages. Such a 
design allowed investigation of task struc >re on performance. Therefore, the principal research questions in this study 
were: 1) what will be the effect of subjects working in pairs on performance on a task involving variables which behave 
in a counterintuitive manner?; and 2) what is the effect of task structure on such performance? 

Methods 



Subjects 

Subjects were 66 students in a teacher education program. Twenty-one students were enrolled in a 1 semester 
hour course titled "Introduction to Microcomputing for Teachers", 30 students were enrolled in a three semester hour 
course titled "Educational Psychology and Measurement", and the remainder (15) were enrolled in a three semester hour 
course titled "A Survey of Computer Applications in Instruction", All were given bonus points for participation. 

The Simulation 

The computer simulation was a model of the spread of an influenza epidemic. There were two displays in the 
simulation (Figure 1 and 2) and it was very easy to operate. On the control display (Figure 1) the subject dragged sliders 
to choose values of the 4 variables. The subject then pressed the Run Simulation button to obtain the next display 
(Figure 2). This display shows both tabular and graphic representation of the number of people ill across time. The 
subject could then click the Simulation Control button (returning to the control display) to try new values and observe 
the output, or click the End of Phase 1 button to go on to the next phase of the experiment. Typically, subjects go back 
and forth between these two displays, changing variables and observing the result, until they believe they have solved the 
assigned problem. All subjects were presented with the following goal: to determine the combination of variable values 
that would keep the maximum number of people ill in any one week as low as possible. Subjects were told that they 
could consider the outbreak to be over as soon as the number of people ill fell below 150 as the outbreak receded. 




573 



4 



Fi gure 1 . Simulation control display. 
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Figure 2 . Graphical and tabular output from simulation. 




In the simulation model, the number of people ill with influenza depended on four variables: the number of 
contacts per person per week, the time to illness, the duration of illness, and the length of the immune period. The 
output observed was the rise and decline of infection within a population of 10,000 people over 32 weeks. The range of 
these variables are presented in Table 1. Table 2 presents the optimal values of these variables for each of the goals 
presented to subjects. In Quinn & Alessi (1994), the proportion of subjects who reached the correct conclusion for Goal 
1 was approximately 70%. In addition, the principal reason for an incorrect conclusion to Goal 1 was an incorrect 
specification for the optimal value of the variable ‘Time to Illness” - 50.6% of subjects who failed to complete the task 
successfully incorrectly specified the minimum value of the variable ‘Time to Illness” rather than the maximum value. 
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Table 1. 



fongcs and of Variables Affecting Spread of Influenza in the 
Simulation M odel, 



Variable 


Range 


Number of Contacts per 
Person per Week 


2 to 10 persons 


Time to Illness 


3 to 28 days 


Duration of Illness 


14 to 35 days 


Length of Immune Period 


13 to 27 weeks 


Table 2. 




Optimum Values of Variables Affecting Spread of Influenza 


in the Simulation Model. 



Variable 


Optimal values for 
minimizing maximum 
number ill in any 
week. 


Optimal values for 
minimizing the number 
of weeks the outbreak 
lasts. 


Number of Contacts per 
Person per Week 


2 persons 


2 persons 


Time to Illness 


28 days 


2 days 


Duration of Illness 


14 days 


14 days 


Length of Immune Period 


27 weeks 


27 weeks 



Design 

Subjects were divided into four treatment groups. In each treatment the overall goal was the same and subjects 
were required to manipulate all four variables. In two groups (1234 and 14) subjects worked on their own and in two 
groups (G234 and G4) students worked in pairs. All subjects were randomly assigned. In groups 14 and G4* subjects 
were presented with all four variables initially. In groups 1234 and G234* subjects were first presented with two variables 
and were required to specify the optimal value of these two variables (Phase 1). When subjects concluded they had 
specified the correct combination of variables* a third variable was introduced and the process was repeated for three 
variables (Phase 2). Finally* a fourth variable was presented and subjects determined the optimal set of values for all four 
variables (Phase 3). For each of the four groups* subjects were asked to complete a questionnaire about their interaction 
with the simulation. The dependent variables measured were the proportion of subjects who reached the correct solution 
(Correct) and the amount of investigation as measured by number of simulation runs performed by subjects (Simrun). 

Procedures 

Subjects in groups 14 and 1234 (who worked individually) first proceeded through an orientation phase where 
they were introduced to the subject matter of the simulation and learned how to manipulate the simulation. In this phase* 
subjects were also given a brief description of the four variables. Subjects were then asked to generate and record an 
initial hypothesis as to what combination of variables would keep the number of weeks the outbreak lasts as small as 
possible* while at the same time keeping the maximum number ill in any one week less than 1500. Subjects could 
specify a range of values for any or all variables and were also given the option of indicating that they did not think that 
there was any combination of variables which would satisfy the goal. Then, subjects were instructed to begin 
investigation by manipulating the levels of the variables and running the simulation. Before each’ run of the simulation* 
subjects were required to record on an Experimentation Record Sheet the levels of each of the variables in the proposed 
simulation run. Then* subjects were instructed to begin investigation by manipulating the levels of the variables and 
running the simulation. After each simulation run* subjects were asked to record the maximum number ill in any one 
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week period and the length of the outbreak. Subjects were not given feedback as to the correctness of the levels chosen. 
Only by manipulating variables and observing the output from the simulation could subjects increase their confidence 
that a particular set of values would generate data to fulfill the goals of the simulation. 

Subjects in groups G4 and G234 proceeded through the task in a manner similar to subjects in groups 14 and 
1234 except that additional instructions were given about working together. In the orientation phase, subjects working in 
pairs were told that they would be working on the task together and were instructed to proceed through the orientation 
phase working together. At this point, subjects were also told that throughout the study, they could discuss as much as 
they wish and perform as many simulation runs as they wished. One Experimentation Record Sheet was available to 
each group. Subjects were told to decide which group member would record data on the record sheet and which member 
would interact with the simulation. Finally, it was emphasized to subjects that while they would work in groups, it was 
not necessary that they agree on the optimal combination of values for the specified goal. Each group member recorded 
their conclusions separately. 



Results 

Analysis of variance was first performed to determine the effectiveness and efficiency of the two treatments - 
Learning Context (individual versus group work), and Task Structure (presentation of variables all at once or in parts). 
This was done by analyzing the dependent variables Correct - proportion of subjects reaching the correct conclusion, and 
Simrun - the number of simulation runs performed by subjects. 

Effectiveness 

The dependent variable. Correct, measured the proportion of subjects who came to the correct conclusion. 
Subjects who came to the correct conclusion were given a score of 1; subjects who did not reach the correct conclusion 
were given a score of 0. Table 3 presents the mean proportion and standard deviation of students who reached the correct 
conclusion in the simulation in each group. Analysis of variance indicated a significant Learning Context by Task 
Structure interaction [F( 1 ,6 1 ) = 5.330, MSerror = 1.224, p = .024], The data are plotted for clarity in Figure 3. In view 
of the interaction, simple effects were examined at each level of Learning Context. The test for simple effects of 
Learning Context indicated a significant effect for Learning Context only in the case where all four variables influencing 
the outcome of the simulation were presented at once [F(l,61) — 20.436, MSerror = 4.694, p = .000]. When all four 
variables were presented at once, subjects performed better when they worked in a group than when they worked alone. 
The test for simple effects of Task Structure indicated a significant effect for Task Structure when subjects worked 
individually [F(l,61) = 38.773, MSerror = .230, p = .000] and in groups [F(l,61) = 8.845, MSerror = .230, p = .004. A 
significantly greater proportion of subjects who worked individually reached the correct conclusion when the task was 
presented in parts; in contrast a greater proportion of subjects who worked in groups reached the correct conclusion when 
the task was presented in its entirety. 



Figure 3 . 



Proportion of subjects reaching the correct 
conclusion by learning context and task structure. 
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Table 3. 

Proportion of Subjects Reachin g the Correct Conclusion. 



GROUP 




Proportion of Subjects Reaching Correct 
Conclusion 


Group 1234 


M 


.444 




SD 


.511 






(n=18) 


Group G234 


M 


.278 




SD 


.461 






(n=18) 


Group 14 


M 


.400 




SD 


.507 






(n=15) 


Group G4 


M 


.786 




SD 


.426 






(n=14) 
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Efficiency 

The dependent variable Simnin measured the number of simulation runs performed by each subject. Table 4 
presents the means and standard deviations for Simrun only for those subjects who successfully completed the 
simulation. Analysis of variance did not indicate any significant differences between groups in terms of efficiency. 

Table 5 presents the means and standard deviations for Simrun for all groups. Analysis of variance indicated 
significant main effects for Task Structure [F( 1,6 1) = 12.644, MSerror = 221.880, p = .001]. Subjects performed 
significantly fewer simulation runs when the task was presented all at once rather than in parts. 

Table 4. 

Number of Simulation Runs Performed by Subjects Reaching the Cores; , Conclusion, 



GROUP 




Number of Simulation Runs Performed by 
Subjects Reaching the Correct Conclusion 


Group 1234 


M 


31.835 




SD 


12.722 






(n=8) 


Group G234 


M 


36.800 




SD 


21.241 






(n=5) 


Group 14 


M 


30.107 




SD 


20.203 






(n=6) 


Group G4 


M 


18.091 




SD 


11.077 






(n=ll) 



Table 5. 

Number of Simula tion Runs Performed by Al l Subjects. 



GROUP 




Number of Simulation Runs 
Performed by All Subjects 


Group 1234 


M 


30.111 




SD 


13.248 






(n=18) 


Group G234 


M 


35.444 




SD 


16.343 






(n=18) _ 


Group 14 


M 


20.400 




SD 


16.008 






(n=15) 


Group G4 


M 


18.714 




SD 


13.652 






12=12 
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Discussion 



With regard to learning effectiveness, the interaction effects indicated no clear advantage for either Learning 
Context or Group Structure alone. Thus, there was no clear advantage for presenting the task in parts or allowing 
subjects to work with a partner. Rather, the results indicated that the effectiveness of allowing subjects to work with a 
partner depended on whether all variables were presented at once or in parts. The findings suggest that subjects who 
worked with a partner performed better when the task was presented all at once than when it was presented in stages. 

This finding suggests that when a task is complex, individuals may benefit from working with a partner, but when a task 
is simple there may be no advantage to working with a partner. On the other hand, subjects who worked alone performed 
better when the task was presented in parts rather than all at once. Thus, our hypothesis that subjects in the group 
condition would outperform those working individually was not confirmed. In addition, there was no clear advantage for 
breaking the task into subtasks. 

It was not surprising that groups performed better when all variables were presented at once. We might have 
concluded that the simple task didn't require the extra resouices of a group if both the individuals and groups had done 
very well, but in fact both did rather poorly, especially the groups. This may be due to the fact that presenting the 
simulation in stages of increasing complexity made it logistically more complicated. Students may have expended a 
great deal of effort on the first sections and then may not have wanted to devote much of their time or attention to the 
final stage to reach a correct conclusion. Groups did not make significantly more runs but they may have taken a bit 
longer per run especially in the simplified presentation. Thus the groups who were presented with the task in parts may 
have become fatigued toward the end of the simulation. 

Another major finding in this study was that subjects who worked individually performed significantly better 
when the task was presented in parts. One implication of this finding is that when subjects work alone they do not have 
access to the types of support provided in a group setting, thus they benefit from instructional support built into the 
instruction. Overall the findings indicate that there are two ways to provide instructional support. First the task may be 
divided into more manageable parts or individuals may be allowed to work with a partner; however both of these 
strategies or types of nondirective support should not be combined. Also, although learning is improved by 
decomposing a task for individuals working alone, the far more effective strategy is to present the task all at once but to 
groups rather than to individuals. 

In terms of learning efficiency, the results of this study indicated an advantage for presenting the task in its 
complexity. Subjects performed significantly fewer runs when they were presented with all four variables at once. This 
finding coupled with the finding regarding learning effectiveness suggests that learning is most effective and efficient 
when tasks are presented in all their complexity to groups as opposed to individuals. One implication of this result is 
that attempting to simplify a task by decomposing it into parts may make the task more logistically cumbersome. 
Another interpretation of this finding is that breaking a complex task into its component parts and making it more 
cognitively manageable for learners may increase their willingness to persevere on a task. 

Conclusion 

This study investigated the effects of group versus individual learning and task presentation format on learners' 
ability to understand and simultaneously consider the effects of more than one variable on a single dependent variable. 
There appears to be no clear advantage for individual versus group work. The effectiveness of either learning context 
depends on the complexity of the task at hand. When the task is simple, learning context does not make a difference in 
performance. However, when the task is complex there is a significant advantage to working in a group. Some of the 
benefits of working with other individuals on a task are that it encourages one to view phenomena from multiple 
perspectives, increases level of commitment to solving a task, and also increases the amount of cognitive effort invested 
in a task because of the subtle social pressure of working in a group (Rysavy & Sales, 1991). Thus, individuals are 
more likely to reap the benefits of working in groups on complex tasks than on simple tasks, since simple tasks do not 
require viewing ideas from multiple perspectives, or investing a great amount of cognitive effort, or an inordinate amount 
of perseverance. The findings of this study suggest that a more effective way of facilitating learning of complex tasks is 
to have individuals work in groups rather than simplify the task by decomposing it into more manageable pieces. This 
finding is cncou r r.ging in light of the practical concerns of most teachers. Rather than spend time watering down difficult 
concepts teachers would do better to simply have students work with a partner to solve a difficult task or understand a 
complex topic. This strategy does not require more time on the part of the teacher and also has numerous other benefits 
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for the student apart from the cognitive advantages, such as the acquisition and practice of social skills, improved self 
esteem, and the increase in perceived status of low achieving students (Rysavy & Sales, 1991). 

Given the increasing emphasis on constructivist software tools such as simulations, it is imperative that 
educators identify ways to help students manage the cognitive demands of computer-based simulations yet still preserve 
their exploratory nature. This study suggests that one way to facilitate learning from simulations which present complex 
tasks is to allow subjects to work in groups. However, this study did not assess comprehension of the material presented 
in the simulation, thus, one issue that needs to be addressed in future research is comprehension as opposed to simply 
arriving at the correct conclusion in a simulation or not. Thus, future research should assess comprehension through the 
use of a posttest given that it is entirely reasonable that subjects may have learned a great deal yet may still not have 
arrived at the correct conclusion. 

A second issue that needs to be addressed in future research concerns the optimal size of groups. At what point 
do the logistics of additional group members outweigh the pedagogical benefits of collaborative learning in a simulation? 
Future research should investigate the effects of allowing subjects to work in groups consisting of more than two 
members. 

A third issue for future research concerns the operational definition of task complexity. In this study, task 
complexity was defined in terms of logistics. That is, the task was simplified by decomposing it into parts as opposed 
to presenting a different task that was conceptually simpler. It would be of interest to investigate the interaction between 
task complexity when the task is made simpler conceptually as well as logistically. 
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